Quick start on watsonx.ai LLM

watsonx.ai introduction

watsonx.ai is IBM's commercial generative AI and scientific data platform based on cloud technology. It encompasses a studio, data store, and governance toolkit, designed to support multiple large language models (LLMs). This platform is tailored for a wide range of AI development tasks, offering developers access to IBM's own series of LLMs as well as models from Facebook’s LLaMA-3 and the Hugging Face community.

Alt text

In this section, we will guide you through the process of using watsonx.ai's API to create a simple Q&A bot. This bot will leverage the advanced capabilities of watsonx.ai to understand and respond to user queries accurately. Whether you're new to programming or an experienced developer, this step-by-step tutorial will equip you with the knowledge to integrate watsonx.ai's LLMs into your applications.

Create a Q&A bot

This tutorial will walk you through the process of creating a Q&A chatbot leveraging the llama-3-2-11b-vision-instruct model developed by Meta. This powerful foundation model has been seamlessly integrated into IBM's watsonx.ai platform, simplifying your development journey. Our provided API eliminates the need for generating complex API tokens, streamlining the application creation process. The llama-3-2-11b-vision-instruct model is equipped with features designed to

  • Supports Q&A
  • Summarization
  • Classification
  • Generation
  • Extraction
  • Retrieval-Augmented Generation tasks

This model aligns perfectly with our objective for the application being created.

Note: The llama-3-2-11b-vision-instruct model, like any AI technology, has its limitations, and it is possible to encounter nonsensical responses occasionally. The primary objective of this project is to provide guidence on utilizing LLMs with watsonx.ai.

Follow these step-by-step instructions to create your application:

  1. Still in the PROJECT directory, create a new Python file named simple_llm.py (you are welcome to choose a different name if you prefer).

  2. Enter the following script content into your newly created simple_llm.py file and save your changes. Line-by-line explanation of the code snippet is provided.

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
  43. 43
  44. 44
  45. 45
  46. 46
  47. 47
  48. 48
  49. 49
  1. # Import necessary packages
  2. from ibm_watsonx_ai import Credentials
  3. from ibm_watsonx_ai import APIClient
  4. from ibm_watsonx_ai.foundation_models import Model, ModelInference
  5. from ibm_watsonx_ai.foundation_models.schema import TextChatParameters
  6. from ibm_watsonx_ai.metanames import GenTextParamsMetaNames
  7. # Model and project settings
  8. model_id = "meta-llama/llama-3-2-11b-vision-instruct" # Directly specifying the LLAMA3 model
  9. # Set credentials to use the model
  10. credentials = Credentials(
  11. url = "https://us-south.ml.cloud.ibm.com",
  12. )
  13. # Set necessary parameters
  14. params = TextChatParameters()
  15. # Specifying project_id as provided
  16. project_id = "skills-network"
  17. # Initialize the model
  18. model = ModelInference(
  19. model_id=model_id,
  20. credentials=credentials,
  21. project_id=project_id,
  22. params=params
  23. )
  24. prompt_txt = "How to be a good Data Scientist?" # Your question
  25. messages = [
  26. {
  27. "role": "user",
  28. "content": [
  29. {
  30. "type": "text",
  31. "text": prompt_txt
  32. },
  33. ]
  34. }
  35. ]
  36. # Attempt to generate a response using the model with overridden parameters
  37. generated_response = model.chat(messages=messages)
  38. generated_text = generated_response['choices'][0]['message']['content']
  39. # Print the generated response
  40. print(generated_text)
  1. Open your terminal and ensure that you are operating within the virtual environment (my_env) you previously established.

  2. Run the following command in the terminal.

  1. 1
  1. python3.11 simple_llm.py

Upon successful execution of the code, a response is displayed in the terminal. An example response follows:

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
  43. 43
  1. '''
  2. To be a good data scientist, you'll need to possess a combination of technical, business, and soft skills. Here are some key areas to focus on:
  3. **Technical Skills:**
  4. 1. **Programming languages:** Proficiency in languages such as Python, R, SQL, and Julia.
  5. 2. **Data manipulation and analysis:** Familiarity with data structures, statistical modeling, and machine learning algorithms.
  6. 3. **Data visualization:** Ability to create interactive and informative visualizations using tools like Tableau, Power BI, or D3.js.
  7. 4. **Big data technologies:** Knowledge of Hadoop, Spark, and NoSQL databases.
  8. 5. **Deep learning:** Familiarity with frameworks like TensorFlow, PyTorch, or Keras.
  9. **Mathematical and Statistical Skills:**
  10. 1. **Linear Algebra:** Understanding of vector spaces, linear transformations, and matrix operations.
  11. 2. **Calculus:** Familiarity with derivatives, integrals, and optimization techniques.
  12. 3. **Statistics:** Knowledge of probability distributions, statistical inference, and hypothesis testing.
  13. 4. **Machine learning theory:** Understanding of the theoretical foundations of machine learning.
  14. **Business Skills:**
  15. 1. **Communication:** Ability to effectively communicate complex technical concepts to non-technical stakeholders.
  16. 2. **Project management:** Experience with project planning, execution, and delivery.
  17. 3. **Business acumen:** Understanding of business objectives, industries, and markets.
  18. 4. **Stakeholder management:** Ability to collaborate with cross-functional teams and stakeholders.
  19. **Soft Skills:**
  20. 1. **Problem-solving:** Ability to approach complex problems in a methodical and creative way.
  21. 2. **Critical thinking:** Capacity to analyze data, identify patterns, and draw meaningful conclusions.
  22. 3. **Collaboration:** Willingness to work with others, share knowledge, and learn from others.
  23. 4. **Adaptability:** Flexibility in adapting to new tools, technologies, and methodologies.
  24. 5. **Continuous learning:** Commitment to staying up-to-date with industry trends, technologies, and best practices.
  25. **Additional Tips:**
  26. 1. **Practice data science:** Apply your skills to real-world problems and projects.
  27. 2. **Read industry blogs and books:** Stay informed about the latest developments in data science and AI.
  28. 3. **Network with professionals:** Attend conferences, meetups, and join online communities to connect with other data scientists.
  29. 4. **Participate in hackathons and competitions:** Showcase your skills and learn from others.
  30. 5. **Pursue certifications:** Consider obtaining certifications like Certified Data Scientist (CDS) or Certified Analytics Professional (CAP) to demonstrate your expertise.
  31. By focusing on these areas, you'll be well on your way to becoming a proficient and effective data scientist.
  32. '''

In the code, we simply used "skills-network" as project_id to gain immediate, free access to the API without the need for initial registration. It's important to note that this access method is exclusive to this Cloud IDE environment. If you are interested in using the model/API in a local environment, detailed instructions and further information are available in this tutorial.

Integrate the application into Gradio

Having successfully created a Q&A bot with our script, you might notice that responses are only displayed in the terminal. You may wonder if it's possible to integrate this application with Gradio to leverage a web interface for inputting questions and receiving responses.

The following code guides you through this integration process. It includes three components:

  • Initializing the model.
  • Defining the function that generates responses from the LLM.
  • Constructing the Gradio interface, enabling interaction with the LLM.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
  33. 33
  34. 34
  35. 35
  36. 36
  37. 37
  38. 38
  39. 39
  40. 40
  41. 41
  42. 42
  43. 43
  44. 44
  45. 45
  46. 46
  47. 47
  48. 48
  49. 49
  50. 50
  51. 51
  52. 52
  53. 53
  54. 54
  55. 55
  56. 56
  57. 57
  58. 58
  59. 59
  1. # Import necessary packages
  2. from ibm_watsonx_ai import Credentials
  3. from ibm_watsonx_ai import APIClient
  4. from ibm_watsonx_ai.foundation_models import Model, ModelInference
  5. from ibm_watsonx_ai.foundation_models.schema import TextChatParameters
  6. from ibm_watsonx_ai.metanames import GenTextParamsMetaNames
  7. import gradio as gr
  8. # Set credentials to use the model
  9. credentials = Credentials(
  10. url = "https://us-south.ml.cloud.ibm.com",
  11. )
  12. # Model and project settings
  13. model_id = "meta-llama/llama-3-2-11b-vision-instruct" # Directly specifying the LLAMA3 model
  14. project_id = "skills-network" # Specifying project_id as provided
  15. params = TextChatParameters(
  16. temperature=0.1,
  17. max_tokens=256
  18. )
  19. # Initialize the model
  20. model = ModelInference(
  21. model_id=model_id,
  22. credentials=credentials,
  23. project_id=project_id,
  24. params=params
  25. )
  26. # Function to generate a response from the model
  27. def generate_response(prompt_txt):
  28. messages = [
  29. {
  30. "role": "user",
  31. "content": [
  32. {
  33. "type": "text",
  34. "text": prompt_txt
  35. },
  36. ]
  37. }
  38. ]
  39. generated_response = model.chat(messages=messages)
  40. generated_text = generated_response['choices'][0]['message']['content']
  41. return generated_text
  42. # Create Gradio interface
  43. chat_application = gr.Interface(
  44. fn=generate_response,
  45. flagging_mode="never",
  46. inputs=gr.Textbox(label="Input", lines=2, placeholder="Type your question here..."),
  47. outputs=gr.Textbox(label="Output"),
  48. title="Watsonx.ai Chatbot",
  49. description="Ask any question and the chatbot will try to answer."
  50. )
  51. # Launch the app
  52. chat_application.launch()
  1. Navigate to the PROJECT directory, right-click and create a new file named llm_chat.py.
  2. Input the script provided above into this new file.
  3. Open your terminal and ensure you are within the my_env virtual environment.
  4. Execute the following code in the terminal to run the application.
  1. 1
  1. python3.11 llm_chat.py

After it has excuted successfully, you will see message similar to the following in the terminal:

Alt text

  1. Click the following button to launch and view the application.

The chatbot you have successfully created will be displayed, appearing as follows:

Alt text

Now, feel free to ask any question to the chatbot.

Here is an example of the question asked:
Alt text

(To terminate the script, press Ctrl+C in the terminal and close the appliction window.)

Exercise

You might observe that the responses from the LLM are occasionally incomplete. Could you identify the cause of this issue? Also, would you be able to modify the code to enable the model to generate more content?

Actually, all you need to do is:

Click here for the answer
  1. 1
  1. "max_tokens": 512, # adjust the parameter `max_token` to a bigger value

In the following section, we will explore how to develop a resume polisher using the knowledge we have just acquired.